Translation of verbal directions into a list of maneuvers

ABSTRACT

Natural language directions are received and a set of maneuver/context pairs are generated based upon the natural language directions. The set of maneuver/context pairs are provided to a routing engine to obtain route information based upon the set of maneuver/context pairs. The route information is provided to an output system for surfacing to a user.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of and claims priority of U.S.patent application Ser. No. 16/196,612, filed Nov. 20, 2018, which is acontinuation of and claims priority of U.S. patent application Ser. No.15/256,834, filed Sep. 6, 2016, the contents of which are herebyincorporated by reference in their entirety.

BACKGROUND

Computer systems are currently in wide use. Some such computer systemsinclude mapping systems. Mapping systems often allow a user to provide astart location and a destination, and the mapping system then calculatesa route between those two geographic locations. The mapping system canalso output a turn-by-turn list of maneuvers for traveling between thestart location and the destination.

Sometimes, however, human beings convey directions or route informationto one another using natural language. The directions or routeinformation may be in colloquial terms which may be relativelyimprecise. Therefore, a computing system may not be able to easilydetermine a route from the natural language directions or routeinformation.

In addition, a human being may know a best route to take (e.g., one thatis better than a route automatically generated by a mapping system).However, it can be difficult to convey that route to another humanbeing. This is sometimes done, currently, by obtaining a default routefrom a mapping system and then providing user inputs to drag the defaultroute provided by the mapping system so that it conforms to the bestroute known by the user. It can also be done by adding artificialwaypoints to force the mapping system to output the route desired by theuser.

The discussion above is merely provided for general backgroundinformation and is not intended to be used as an aid in determining thescope of the claimed subject matter.

SUMMARY

Natural language instructions are received and a set of maneuver/contextpairs are generated based upon the natural language instructions. Theset of maneuver/context pairs are provided to a routing engine to obtainroute information based upon the set of maneuver/context pairs. Theroute information is provided to an output system for surfacing to auser.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter. The claimed subject matter is not limited to implementationsthat solve any or all disadvantages noted in the background.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one example of a computing systemarchitecture.

FIG. 2 is a block diagram showing one example of trigger identificationlogic (shown in FIG. 1 ) in more detail.

FIG. 3 is a flow diagram illustrating one example of the operation ofthe architecture shown in FIG. 1 and the trigger identification logicshown in FIG. 2 in generating route information from a set of naturallanguage directions.

FIG. 4 is a block diagram of one example of the architecture illustratedin FIG. 1 , deployed in a cloud computing architecture.

FIGS. 5-7 show examples of mobile devices that can be used in thearchitectures shown in the previous figures.

FIG. 8 is a block diagram of one example of a computing environment thatcan be used in the architectures shown in the previous figures.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of one example of a computing systemarchitecture 100. Architecture 100 includes computing system 102 thatgenerates user interfaces 104 with user input/output mechanisms 106 forinteraction by user 108. User 108 can interact with the userinput/output mechanisms 106 in order to control and manipulate computingsystem 102. FIG. 1 also shows that, in architecture 100, computingsystem 102 can be coupled to access remote systems or services 110 usingnetwork 112. Network 112 can be any of a variety of different types ofnetworks, such as a wide area network, a local area network, a cellularnetwork, a near field communication network, among others.

In the example shown in FIG. 1 , computing system 102 illustrativelyincludes one or more processors or servers 114, speechrecognition/natural language processing system 116, data store 118,standard maneuver generator system 120, routing engine 122, speechsynthesis system 124, output system 126, user interface logic 128, andit can include a variety of other computing system functionality 130.Standard maneuver generator system 120, itself, illustratively includesstart/end point identifier logic 132, trigger identification logic 134,disambiguation system 136, standard maneuver output logic 138, and itcan include other items 140. Data store 118 can, itself, includeuser-specific location information, such as information identifying userfavorites 142, user preferences 144, and a wide variety of other items146. Before describing the overall operation of computing system 102 inmore detail, a brief overview of some of the items in architecture 100,and their operation, will first be provided.

Speech recognition/natural language processing system 116 can be locatedon computing system 102 or remotely from computing system 102 andaccessed by computing system 102. In one example, system 116illustratively receives a natural language input, which can be a speechinput, a written textual input, etc. If it is a speech input, the speechis recognized by the speech recognition portion of system 116 so thatwords (or other linguistic units) in the speech input are identified.The natural language processing portion of system 116 then performsnatural language processing on those linguistic units (or on linguisticunits in a natural language input that is provided in textual form) toidentify a semantic meaning, or linguistic meaning, of the units. Theseare provided as natural language directions 150, to standard maneuvergenerator system 120.

Trigger identification logic 134 then identifies a set ofmaneuver/context pairs. These pairs can be referred to as triggers. Themaneuver/context pairs can also include limiters as well. For example,the natural language directions 150 may be “stay on Pine Street untilyou get to the coffee shop, and then turn left onto Broadway.” Triggeridentification logic 134 may identify, in that input, a maneuver such as“turn [direction]”, where the direction is “left”. It may also identifycontext information which indicates a location where the left turn is tobe taken. In this example, the context information may be “at the coffeeshop, onto Broadway”. Disambiguation system 136 then uses other contextinformation in the natural language directions 150 to identify whichcoffee shop (or a geographic location of the coffee shop) the user isreferring to. For instance, disambiguation system 136 may identify acoffee shop at the intersection of Pine Street and Broadway as “ACMECoffee Shop”. Thus, trigger identification logic 134 may generate, as amaneuver/context pair, “turn left onto Broadway at the ACME Coffee Shoplocated at 123 Pine Street”.

In one example, trigger identification logic 134 can also identifylimiters. For instance, if the natural language direction 150 alsoincluded the directions “if you reach the fast food restaurant, you'vegone too far”. In that case, disambiguation system 136 identifies theparticular “fast food restaurant” (or its geographic location) givenother context information, and trigger identification logic 134 cangenerate the maneuver/context pair, including a limiter, as a trigger.

It will also be noted, in one example, start/end point identifier logic132 may identify a start point and a destination, based upon the naturallanguage directions 150, which may be relatively imprecise with respectto those locations. For instance, it may be that natural languagedirections 150 begin with a text string such as “get on 520 headingEast”. In that case, start/end point identifier logic 132 identifies anorigin, such as the user's current location, and identifiesmaneuver/context pairs that take the user from the identified origin to“the 520”.

The trigger identification logic 134 can then generate triggers,including the maneuver/context pairs from the origin along with anylimiters, and provide them to standard maneuver output logic 138. Logic138 illustratively outputs a sequentially ordered list of triggers(comprising standard maneuvers corresponding to the natural languagedirections 150) in the form of the maneuver/context (and possiblylimiter) pairs. This is indicated by block 152 in FIG. 1 .

These are provided to routing engine 122 which takes the sequentialtriggers (e.g., the sequential maneuver/context pairs) as start and endpoints and then calculates a route between those two points. It can dothis using route criteria 154 to identify the particular route. Theroute criteria may include such things as “shortest distance”, “use mainroads”, “add as few additional waypoints as possible”, etc. In oneexample, routing engine 122 calculates the route to conform as closelyas possible, given the route criteria 154, to the route represented bythe ordered list of triggers 152. It outputs route information 156indicative of the route represented by triggers 152. Thus, the routeinformation 156 conforms closely to the natural language directions 150received by system 120. In this way, it is contemplated that the userthat provided the natural language directions 150 may know a best routefor another user to take. Thus, system 102 calculates route information156 to conform to the natural language directions 150 provided,initially, by the user.

In one example, speech synthesis system 124 can be used to generate anaudible, verbal output that represents the route information 156. Thisoutput may provide instructions or directions to the user in order tonavigate the route. It will be noted that the verbal output can take avariety of different forms. It can be part of an existing navigationsystem (such as guided directions that tell the user what to do). It canalso be generated by taking the corresponding snippet from the originalrecording that matches the trigger and playing that at the point in thenavigation corresponding to that trigger. This way, a user who got thedirections from a friend can hear the instructions in their friend'svoice.

In another example, the snippets from the original recordings can beinterspersed with standard guided navigation recordings. Or, in anotherexample, the verbal instructions include only the snippets from theoriginal recording and the user can rely on the visually displayeddirections for portions of the route that lie between the snippets.

The route information (and possibly corresponding speech synthesisinformation) can be provided to any of a variety of different outputsystems 126 for surfacing to user 108. In one example, output system 126is a mapping application which displays the route information 156 as amaneuvers list that user 108 can follow in order to navigate the route.In another example, output system 126 is a guided navigation system thatwalks the user through each step in the route information 156, to guidethe user through navigation of the route. These outputs can be in theform of a display, they can be audible, they can be other outputs orcombinations of outputs. In another example, output system 126 can be alocal or remote storage system where the route information 156 can besaved and shared.

FIG. 2 is a block diagram showing one example of trigger identificationlogic 134 in more detail. In the example shown in FIG. 2 , logic 134illustratively includes maneuver identifier logic 160, contextidentifier logic 162, limiter identifier logic 164, trigger output logic166, and it can include a variety of other items 168. Maneuveridentifier logic 160 illustratively parses the natural languagedirections 150 to match portions of the directions 150 with knownnavigation maneuvers, such as “turn right on Pine Street”. Contextidentifier logic 162 identifies context information (such as locationinformation) corresponding to each identified maneuver. Limiteridentifier logic 164 illustratively identifies any limiterscorresponding to the maneuver/context pair identified by logic 160 and162. Trigger output logic 166 can output a trigger in the form of themaneuver/context pair, along with any limiters identified by logic 164.In one example, logic 160, logic 162, and logic 164 are natural languageprocessing components that identify the standard maneuvers, the context(or location) corresponding to the standard maneuvers and any limitersusing natural language processing techniques. They can be rules basedsystems in which items in natural language directions 150 are matchedagainst rules and matching rules are triggered to identify maneuversand/or context information, and possibly limiters. Logic 160, 162 and164 can also be implemented using model-based natural languageprocessing systems, heuristically-based systems, or other systems.

Some examples of the maneuvers that maneuver identifier logic 160 canidentify include those set out below in Table 1. These are examplesonly.

TABLE 1 ″Turn″ [direction] ″at the″ [place | object] {optional: ″on the″[direction]} ″After the″ [place | object] {optional: ″on the″[direction]} ″Pass the″ [place | object] ″Before the″ [place | object]″Pretend like you are going to″ [place] place = searchable location(ACME, hotel, gas station, etc.) object = light | stop sign | firehydrant | bushes | parking lot

The triggers output by trigger output logic 166 can be disambiguated bydisambiguation system 136, to the extent needed or desired. They canthen be output using logic 138 (shown in FIG. 1 ) as the ordered list ofstandard maneuvers and corresponding context information correspondingto the natural language directions 150.

FIG. 3 is a flow diagram illustrating one example of the operation ofcomputing system 102 and trigger identifier logic 134, in generatingroute information based on natural language directions received by thesystem. In one example, computing system 102 first identifies adestination that is to be used for generating a route. This is indicatedby block 180. A user may provide a natural language input such as “Getdirections to ACME store in Redmond.” Disambiguation system 136 mayperform disambiguation to identify which, particular ACME store this is,if needed. This is indicated by block 182. System 116, or anothersystem, may also search for the ACME store location in data store 118 orin another data store or in a remote system or service 110. This isindicated by block 184. The destination can be identified in other waysas well.

Once the destination is identified, if natural language directionsalready exist from the user's current location to the destination, theycan be retrieved for processing. If not, it may be that the user wishesto record directions to the destination (such as for sharing withanother user). If that is the case, the user can record the directionssuch as by using a microphone or other user input or dictation system.This is indicated by blocks 188 and 190.

At some point, system 102 will detect an input indicating that a set ofnatural language directions 150 is available to be processed. This isindicated by block 200 in the flow diagram of FIG. 3 . The input cantake a wide variety of different forms. For instance, user 108 canactuate a user input mechanism 106 to indicate this. In another example,it may be that standard maneuver generator system 120 receives anautomated indication that a set of natural language directions 150 is tobe processed.

It will also be noted that the natural language directions can be in theform of speech 202, text 204, or they can take other forms 206. Wherethe natural language directions are in the form of speech 202, they maybe in the form of recorded or live speech that is being input by user108. In either case, they are illustratively provided to a speechrecognition system 116 that recognizes words in the speech, or mayrecognize other linguistic units as well. Natural language processingsystem 116 then performs natural language processing to identifydirections in the natural language input. Performing natural languageprocessing to identify directions in the natural language input isindicated by block 208 in the flow diagram of FIG. 3 . natural languagedirections 150 are provided to standard maneuver generator system 120.

Maneuver identifier logic 160 in trigger identification logic 134 thenmatches the natural language directions with known navigation maneuvers.The set of known navigation maneuvers against which the natural languagedirections are matched may be accessed in a data store that is used by arouting engine, a mapping system, or other components or logic. Matchingthe directions with known navigation maneuvers is indicated by block 210in the flow diagram of FIG. 3 .

By way of example, it may be that a natural language input is “make aright”. This may be matched by maneuver identification logic 160 to aknown maneuver of “turn right”. The matching can be performed by using alanguage model, a context free grammar or other grammar, a rules-basedor heuristic-based system, or in other ways.

Context identifier logic 162 then matches each maneuver identified bylogic 160 with corresponding context information. This is indicated byblock 212. In one example, the corresponding context information islocation information 214, indicating a location at which the maneuver isto be performed. However, it can take other forms as well, as indicatedby block 216. As an example, it may be that the maneuver is “turn right”and the context information is “at the intersection of Pine Street andBroadway”. However, the location information may take other forms. Forinstance, instead of “at the corner of Pine Street and Broadway”, thecontext information may be “at the next gas station”. These are examplesonly.

Limiter identifier logic 164 then identifies any limiters that go alongwith the maneuver/context pair and matches the maneuver/context pairwith that limiter. This is indicated by block 218 in FIG. 3 . By way ofexample, it may be that the maneuver is “turn right”, the contextinformation is “at the gas station”, and the limiter is “after the forkin the road”.

In this example, the trigger that comprises the maneuver/context pairwith its limiter would be “turn right at the gas station after the forkin the road.” The various portions of the trigger (maneuver, context,limiter) can be labeled or otherwise marked.

It can be seen that some portions of this may be ambiguous. For example,“the gas station” and “after the fork in the road” may both beambiguous, and may even be ambiguous when taken together. Therefore,trigger identification logic 134 accesses disambiguation system 136which performs any disambiguation processing to disambiguate the contextinformation or the limiter, or both. Performing disambiguation isindicated by block 220 in the flow diagram of FIG. 3 . Thedisambiguation system 136 illustratively considers context informationfrom not only the current maneuver/context pair, but also from othermaneuver/context pairs that appear before or after the current pair, inthe ordered list of triggers. For instance, if the previousmaneuver/context pair was “turn right from Pine Street onto Broadway”,then the disambiguation system 136 can use that information to determinewhich particular gas station corresponds to “the next gas station” inthe maneuver/context pair being disambiguated. It can access locationinformation for all gas stations on “Broadway” and then identify thefirst gas station after the traveler has turned right onto Broadway,from Pine Street, and augment the context information to show theparticular name of that gas station, if desired. It can also replace oraugment the context information and limiter with other identifyinginformation such as geographic location information, addressinformation, or other information.

Start/end point identifier logic 132 then identifies the origin (orstarting point) for user 108. This is indicated by block 224 in the flowdiagram of FIG. 3 .

For instance, if the first natural language direction is “start bygetting on 520 at Broadway Street,” there may be multiple steps to getfrom the user's current location to Broadway Street. However, givencontextual information about the user's current location (e.g., his orher current GPS location, current home address, etc.) and the locationof the closest entry onto Broadway Street, and additional contextualknowledge about the way the user should go to get to the finaldestination, routing engine 122 can generate turn-by-turn directions toget the user into a position where the natural language directionsactually begin.

Trigger output logic 166 then outputs the origin and disambiguatedtriggers to the routing engine 122. This is indicated by block 226 inFIG. 3 . In one example, the ordered list of triggers comprise anordered list of standard maneuvers and corresponding contextinformation. This is provided as a list of maneuvers that are to besequentially performed at individual points identified by the context orlocation information. Routing engine 122 then accesses route criteria154 and calculates a route from the origin to the location of the firsttrigger. It then calculates a route between subsequent pairs ofsequential triggers. This allows the routing engine to generate a routefrom the origin to the destination, while conforming the particulardirections that were input by the user in natural language form. Thus,routing engine 122 does not simply discard the user's directions andcalculate a route form the origin to the destination. Instead, itcalculates that route using (or at least taking into account) thevarious directions that the user input in natural language form. This isbecause it considers the triggers (e.g., maneuver/context pairs—with anydelimiters) in generating the route. Calculating a route from the originto the first trigger and then between all subsequent pairs of triggersto obtain a route from the origin to the destination, using the user'snatural language directions, is indicated by block 228 in FIG. 3 .

The route information 156 can then be output by output system 126 forsurfacing to user 108. This is indicated by block 230 in the flowdiagram of FIG. 3 . In one example, speech synthesis system 124 cangenerate audible directions that accompany the route information 156. Inthat example, the route information can be displayed as a set ofstep-by-step directions, but it can also take the form of spokendirections instead, or as well. Outputting the spoken, audibleinstructions is indicated by block 232.

The route information 156 can also be surfaced as an ordered list ofdirections in a mapping application. This is indicated by block 234. Itcan be used in a guided navigation system to allow user 108 to perform aguided navigation. This is indicated by block 236. It can also be saved(such as at data store 118, at a remote system or service 110, orelsewhere) for later use or sharing. This is indicated by block 238. Theinformation can be output in other ways as well, and this is indicatedby block 240.

It can thus be seen that the present system provides a number ofsignificant advantages. It matches verbal strings in a recording, a liveaudio stream, or even a phone/VOIP call to standard navigation/maneuverterms. It also recognizes locations in verbal instructions, and mapsthem to physical locations. For example, where an instruction is to turnat the coffee shop onto “Greenlake Avenue”, but there are multipleGreenlake Avenues, the present system disambiguates that term to findthe correct Greenlake Avenue, given that the original location anddestination are both in a particular city. It can also identify thecorrect coffee shop to turn at, given that the previous direction was to“turn on Pine Street”. Further, a particular road (such as State HighwayWA-520) may be colloquially referred to by a short name (such as “520”or “the 520”). The present system recognizes that, given the surroundingcontext information.

The present system can also disambiguate specific locations, again basedon secondary information or the context information or limiters in othertriggers. It can also identify or disambiguate attributes of a road byrelating it to mapping data. For instance, the term “the large bend inthe road” or “after the fork” may be used when a physical location isnot the best indicator of when a maneuver should take place.

The triggers use a maneuver (e.g., “turn right”) within a context (suchas “at a location” “or” “at the next gas station”, etc.) to compose amaneuver with a map location. This is illustratively done by searchingfor map locations with the given name (e.g., the name of the gasstation) along the street the user is on, or by interpolating othernearby options based upon the computer-generated route information.

The limiters provide a bound on directions to reduce the need todisambiguate vague or incomplete triggers. For instance, assume that thetrigger is “turn right at the gas station”, and includes the limiter “ifyou have reached the grocery store, you have gone too far.” This type oflimiter provides additional context to the standard maneuver generatorsystem 120 to limit the need for additional user input. Limiters canalso indicate a minimum or maximum distance or time to travel (e.g.,“drive for about 10 minutes” or “head down that road for a couplemiles”). These are used to reduce any need for additional user inputs aswell.

The present system also uses routing engine 122 to provide directionsthat may be used to fill gaps in the verbal directions. For instance, ifthe initial direction is “start by getting on 520 at Montlake”, theremay be multiple steps to get from the user's current location to theMontlake entrance. However, given the user's home address and thelocation of the Montlake entrance ramp, and contextual knowledge aboutwhich way the user should go to get to the final destination, therouting engine 122 can use the start point identified by start/end pointidentifier logic 132 to fill in the gaps to get turn-by-turn directionsfrom the user's current location to the first identified location in thedirections (the entrance ramp to 520 at Montlake).

The same type of processing can be used to fill in gaps in the middle ofthe route. For instance, if the natural language direction is “pretendlike you're driving from Seattle to work, but keep going on 520 untilyou get to the exit you′d use to go to the Target in Redmond”, thetriggers that are generated and provided to routing engine 122 allowrouting engine 122 to translate this into turn-by-turn directions, firstfrom the user to their work, but instead of getting off at 520 to go towork, that location is used as a waypoint. Engine 122 then getsdirections from that waypoint (on 520) to the nearest Acme Store inRedmond.

All of these greatly enhance the operation of the computing system andgreatly reduce the need for additional user input. It also allows thesystem to follow the directions provided by the user in the naturallanguage instructions, and not to simply calculate its own route.Instead, the route that is generated conforms to the route input by theuser is used.

It will be noted that the above discussion has described a variety ofdifferent systems, components and/or logic. It will be appreciated thatsuch systems, components and/or logic can be comprised of hardware items(such as processors and associated memory, or other processingcomponents, some of which are described below) that perform thefunctions associated with those systems, components and/or logic. Inaddition, the systems, components and/or logic can be comprised ofsoftware that is loaded into a memory and is subsequently executed by aprocessor or server, or other computing component, as described below.The systems, components and/or logic can also be comprised of differentcombinations of hardware, software, firmware, etc., some examples ofwhich are described below. These are only some examples of differentstructures that can be used to form the systems, components and/or logicdescribed above. Other structures can be used as well.

The present discussion has mentioned processors and servers. In oneembodiment, the processors and servers include computer processors withassociated memory and timing circuitry, not separately shown. They arefunctional parts of the systems or devices to which they belong and areactivated by, and facilitate the functionality of the other componentsor items in those systems.

Also, a number of user interface displays have been discussed. They cantake a wide variety of different forms and can have a wide variety ofdifferent user actuatable input mechanisms disposed thereon. Forinstance, the user actuatable input mechanisms can be text boxes, checkboxes, icons, links, drop-down menus, search boxes, etc. They can alsobe actuated in a wide variety of different ways. For instance, they canbe actuated using a point and click device (such as a track ball ormouse). They can be actuated using hardware buttons, switches, ajoystick or keyboard, thumb switches or thumb pads, etc. They can alsobe actuated using a virtual keyboard or other virtual actuators. Inaddition, where the screen on which they are displayed is a touchsensitive screen, they can be actuated using touch gestures. Also, wherethe device that displays them has speech recognition components, theycan be actuated using speech commands.

A number of data stores have also been discussed. It will be noted theycan each be broken into multiple data stores. All can be local to thesystems accessing them, all can be remote, or some can be local whileothers are remote. All of these configurations are contemplated herein.

Also, the figures show a number of blocks with functionality ascribed toeach block. It will be noted that fewer blocks can be used so thefunctionality is performed by fewer components. Also, more blocks can beused with the functionality distributed among more components.

FIG. 4 is a block diagram of architecture 100, shown in FIG. 1 , exceptthat its elements are disposed in a cloud computing architecture 500.Cloud computing provides computation, software, data access, and storageservices that do not require end-user knowledge of the physical locationor configuration of the system that delivers the services. In variousembodiments, cloud computing delivers the services over a wide areanetwork, such as the internet, using appropriate protocols. Forinstance, cloud computing providers deliver applications over a widearea network and they can be accessed through a web browser or any othercomputing component. Software or components of architecture 100 as wellas the corresponding data, can be stored on servers at a remotelocation. The computing resources in a cloud computing environment canbe consolidated at a remote data center location or they can bedispersed. Cloud computing infrastructures can deliver services throughshared data centers, even though they appear as a single point of accessfor the user. Thus, the components and functions described herein can beprovided from a service provider at a remote location using a cloudcomputing architecture. Alternatively, they can be provided from aconventional server, or they can be installed on client devicesdirectly, or in other ways.

The description is intended to include both public cloud computing andprivate cloud computing. Cloud computing (both public and private)provides substantially seamless pooling of resources, as well as areduced need to manage and configure underlying hardware infrastructure.

A public cloud is managed by a vendor and typically supports multipleconsumers using the same infrastructure. Also, a public cloud, asopposed to a private cloud, can free up the end users from managing thehardware. A private cloud may be managed by the organization itself andthe infrastructure is typically not shared with other organizations. Theorganization still maintains the hardware to some extent, such asinstallations and repairs, etc.

In the example shown in FIG. 4 , some items are similar to those shownin FIG. 1 and they are similarly numbered. FIG. 4 specifically showsthat computing system 102 can be located in cloud 502 (which can bepublic, private, or a combination where portions are public while othersare private). Therefore, user 108 uses a user device 504 to access thosesystems through cloud 502.

FIG. 4 also depicts another example of a cloud architecture. FIG. 4shows that it is also contemplated that some elements of computingsystem 102 can be disposed in cloud 502 while others are not. By way ofexample, data store 118 can be disposed outside of cloud 502, andaccessed through cloud 502. In another example, standard maneuvergenerator system 120 can be outside of cloud 502. Regardless of wherethey are located, they can be accessed directly by device 504, through anetwork (either a wide area network or a local area network), they canbe hosted at a remote site by a service, or they can be provided as aservice through a cloud or accessed by a connection service that residesin the cloud. All of these architectures are contemplated herein.

It will also be noted that architecture 100, or portions of it, can bedisposed on a wide variety of different devices. Some of those devicesinclude servers, desktop computers, laptop computers, tablet computers,or other mobile devices, such as palm top computers, cell phones, smartphones, multimedia players, personal digital assistants, etc.

FIG. 5 is a simplified block diagram of one illustrative example of ahandheld or mobile computing device that can be used as a user's orclient's hand held device 16, in which the present system (or parts ofit) can be deployed. FIGS. 6-8 are examples of handheld or mobiledevices.

FIG. 5 provides a general block diagram of the components of a clientdevice 16 that can run components of computing system 102 or thatinteracts with architecture 100, or both. In the device 16, acommunications link 13 is provided that allows the handheld device tocommunicate with other computing devices and under some embodimentsprovides a channel for receiving information automatically, such as byscanning. Examples of communications link 13 include an infrared port, aserial/USB port, a cable network port such as an Ethernet port, and awireless network port allowing communication though one or morecommunication protocols including General Packet Radio Service (GPRS),LTE, HSPA, HSPA+ and other 3G and 4G radio protocols, 1Xrtt, and ShortMessage Service, which are wireless services used to provide cellularaccess to a network, as well as Wi-Fi protocols, and Bluetooth protocol,which provide local wireless connections to networks.

In other examples, applications or systems are received on a removableSecure Digital (SD) card that is connected to a SD card interface 15. SDcard interface 15 and communication links 13 communicate with aprocessor 17 (which can also embody processors or servers 114 from FIG.1 ) along a bus 19 that is also connected to memory 21 and input/output(I/O) components 23, as well as clock 25 and location system 27.

I/O components 23, in one embodiment, are provided to facilitate inputand output operations. I/O components 23 for various embodiments of thedevice 16 can include input components such as buttons, touch sensors,multi-touch sensors, optical or video sensors, voice sensors, touchscreens, proximity sensors, microphones, tilt sensors, and gravityswitches and output components such as a display device, a speaker, andor a printer port. Other I/O components 23 can be used as well.

Clock 25 illustratively comprises a real time clock component thatoutputs a time and date. It can also, illustratively, provide timingfunctions for processor 17.

Location system 27 illustratively includes a component that outputs acurrent geographical location of device 16. This can include, forinstance, a global positioning system (GPS) receiver, a LORAN system, adead reckoning system, a cellular triangulation system, or otherpositioning system. It can also include, for example, mapping softwareor navigation software that generates desired maps, navigation routesand other geographic functions.

Memory 21 stores operating system 29, network settings 31, applications33, application configuration settings 35, data store 37, communicationdrivers 39, and communication configuration settings 41. Memory 21 caninclude all types of tangible volatile and non-volatilecomputer-readable memory devices. It can also include computer storagemedia (described below). Memory 21 stores computer readable instructionsthat, when executed by processor 17, cause the processor to performcomputer-implemented steps or functions according to the instructions.Similarly, device 16 can have a client system 24 which can run variousbusiness applications or embody parts or all of tenant 104. Processor 17can be activated by other components to facilitate their functionalityas well.

Examples of the network settings 31 include things such as proxyinformation, Internet connection information, and mappings. Applicationconfiguration settings 35 include settings that tailor the applicationfor a specific enterprise or user. Communication configuration settings41 provide parameters for communicating with other computers and includeitems such as GPRS parameters, SMS parameters, connection user names andpasswords.

Applications 33 can be applications that have previously been stored onthe device 16 or applications that are installed during use, althoughthese can be part of operating system 29, or hosted external to device16, as well.

FIG. 6 shows one example in which device 16 is a tablet computer 600. InFIG. 6 , computer 600 is shown with user interface display screen 602.Screen 602 can be a touch screen (so touch gestures from a user's fingercan be used to interact with the application) or a pen-enabled interfacethat receives inputs from a pen or stylus. It can also use an on-screenvirtual keyboard. Of course, it might also be attached to a keyboard orother user input device through a suitable attachment mechanism, such asa wireless link or USB port, for instance. Computer 600 can alsoillustratively receive voice inputs as well.

FIG. 7 shows that the device can be a smart phone 71. Smart phone 71 hasa touch sensitive display 73 that displays icons or tiles or other userinput mechanisms 75. Mechanisms 75 can be used by a user to runapplications, make calls, perform data transfer operations, etc. Ingeneral, smart phone 71 is built on a mobile operating system and offersmore advanced computing capability and connectivity than a featurephone.

Note that other forms of the devices 16 are possible.

FIG. 8 is one example of a computing environment in which architecture100, or parts of it, (for example) can be deployed. With reference toFIG. 8 , an example system for implementing some embodiments includes ageneral-purpose computing device in the form of a computer 810.Components of computer 810 may include, but are not limited to, aprocessing unit 820 (which can comprise processors or servers fromprevious Figures), a system memory 830, and a system bus 821 thatcouples various system components including the system memory to theprocessing unit 820. The system bus 821 may be any of several types ofbus structures including a memory bus or memory controller, a peripheralbus, and a local bus using any of a variety of bus architectures. By wayof example, and not limitation, such architectures include IndustryStandard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus,Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA)local bus, and Peripheral Component Interconnect (PCI) bus also known asMezzanine bus. Memory and programs described with respect to FIG. 1 canbe deployed in corresponding portions of FIG. 8 .

Computer 810 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 810 and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media is different from, anddoes not include, a modulated data signal or carrier wave. It includeshardware storage media including both volatile and nonvolatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by computer 810. Communication media typically embodiescomputer readable instructions, data structures, program modules orother data in a transport mechanism and includes any informationdelivery media. The term “modulated data signal” means a signal that hasone or more of its characteristics set or changed in such a manner as toencode information in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of any of the aboveshould also be included within the scope of computer readable media.

The system memory 830 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 831and random access memory (RAM) 832. A basic input/output system 833(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 810, such as during start-up, istypically stored in ROM 831. RAM 832 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 820. By way of example, and notlimitation, FIG. 8 illustrates operating system 834, applicationprograms 835, other program modules 836, and program data 837.

The computer 810 may also include other removable/non-removablevolatile/nonvolatile computer storage media. By way of example only,FIG. 8 illustrates a hard disk drive 841 that reads from or writes tonon-removable, nonvolatile magnetic media, and an optical disk drive 855that reads from or writes to a removable, nonvolatile optical disk 856such as a CD ROM or other optical media. Other removable/non-removable,volatile/nonvolatile computer storage media that can be used in theexemplary operating environment include, but are not limited to,magnetic tape cassettes, flash memory cards, digital versatile disks,digital video tape, solid state RAM, solid state ROM, and the like. Thehard disk drive 841 is typically connected to the system bus 821 througha non-removable memory interface such as interface 840, and optical diskdrive 855 are typically connected to the system bus 821 by a removablememory interface, such as interface 850.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Program-specific Integrated Circuits (ASICs), Program-specificStandard Products (ASSPs), System-on-a-chip systems (SOCs), ComplexProgrammable Logic Devices (CPLDs), etc.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 8 provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 810. In FIG. 8 , for example, hard disk drive 841 isillustrated as storing operating system 844, application programs 845,other program modules 846, and program data 847. Note that thesecomponents can either be the same as or different from operating system834, application programs 835, other program modules 836, and programdata 837. Operating system 844, application programs 845, other programmodules 846, and program data 847 are given different numbers here toillustrate that, at a minimum, they are different copies.

A user may enter commands and information into the computer 810 throughinput devices such as a keyboard 862, a microphone 863, and a pointingdevice 861, such as a mouse, trackball or touch pad. Other input devices(not shown) may include a joystick, game pad, satellite dish, scanner,or the like. These and other input devices are often connected to theprocessing unit 820 through a user input interface 860 that is coupledto the system bus, but may be connected by other interface and busstructures, such as a parallel port, game port or a universal serial bus(USB). A visual display 891 or other type of display device is alsoconnected to the system bus 821 via an interface, such as a videointerface 890. In addition to the monitor, computers may also includeother peripheral output devices such as speakers 897 and printer 896,which may be connected through an output peripheral interface 895.

The computer 810 is operated in a networked environment using logicalconnections to one or more remote computers, such as a remote computer880. The remote computer 880 may be a personal computer, a hand-helddevice, a server, a router, a network PC, a peer device or other commonnetwork node, and typically includes many or all of the elementsdescribed above relative to the computer 810. The logical connectionsdepicted in FIG. 8 include a local area network (LAN) 871 and a widearea network (WAN) 873, but may also include other networks. Suchnetworking environments are commonplace in offices, enterprise-widecomputer networks, intranets and the Internet.

When used in a LAN networking environment, the computer 810 is connectedto the LAN 871 through a network interface or adapter 870. When used ina WAN networking environment, the computer 810 typically includes amodem 872 or other means for establishing communications over the WAN873, such as the Internet. The modem 872, which may be internal orexternal, may be connected to the system bus 821 via the user inputinterface 860, or other appropriate mechanism. In a networkedenvironment, program modules depicted relative to the computer 810, orportions thereof, may be stored in the remote memory storage device. Byway of example, and not limitation, FIG. 8 illustrates remoteapplication programs 885 as residing on remote computer 880. It will beappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

It should also be noted that the different embodiments described hereincan be combined in different ways. That is, parts of one or moreembodiments can be combined with parts of one or more other embodiments.All of this is contemplated herein.

Example 1 is a computing system, comprising:

maneuver identification logic that receives a natural languageinstruction and matches it with a pre-defined navigation maneuver;

context identifier logic that identifies context informationcorresponding to the pre-defined navigation maneuver, the contextinformation being indicative of a location at which the pre-definednavigation maneuver is performed;

trigger output logic that outputs, as a trigger, the pre-definednavigation maneuver and the corresponding context information; and

an output system that surfaces the trigger.

Example 2 is the computing system of any or all previous examples andfurther comprising:

limiter identifier logic that identifies any geographic limiter in thenatural language instruction, the trigger output logic being configuredto output the geographic limiter, along with the pre-defined navigationmaneuver and the corresponding context information, as the trigger.

Example 3 is the computing system of any or all previous examples andfurther comprising:

a disambiguation system that performs disambiguation of the contextinformation and the geographic limiter to identify geographic locationscorresponding to the context information and the geographic limiter.

Example 4 is the computing system of any or all previous examples andfurther comprising:

a routing engine that identifies a route based on a first point and asecond point.

Example 5 is the computing system of any or all previous exampleswherein the maneuver identification logic is configured to receive aplurality of natural language instructions and match each with apre-defined navigation maneuver, the context identifier logic beingconfigured to identify context information corresponding to each of thepre-defined navigation maneuvers, the trigger output logic beingconfigured to output, as a set of ordered triggers, a set of pre-definednavigation maneuvers and corresponding context information provided bythe maneuver identification logic and the context identifier logic.

Example 6 is the computing system of any or all previous exampleswherein the routing engine is configured to receive the set of orderedtriggers and generate route information identifying a route betweensuccessive pairs of triggers in the set of ordered triggers.

Example 7 is the computing system of any or all previous examples andfurther comprising:

start point identifier logic that identifies a start point from theplurality of natural language instructions and from user-specificlocation information.

Example 8 is the computing system of any or all previous exampleswherein the routing engine is configured to generate route informationidentifying a route from the start point to a first of the set ofordered triggers.

Example 9 is the computing system of any or all previous examples andfurther comprising:

a speech system that generates navigation speech corresponding to therouting information and outputs the navigation speech.

Example 10 is the computing system of any or all previous exampleswherein the output system surfaces the routing information on anavigation display as a set of turn-by-turn navigation instructions andthe navigation speech on an audio output device as a set of spokennavigation instructions.

Example 11 is the computer system of any or all previous exampleswherein the plurality of natural language instructions are speechinstructions and wherein the speech system generates the navigationspeech by matching the speech instructions in the natural languageinstructions with the corresponding trigger.

Example 12 is the computing system of any or all previous exampleswherein the speech system generates at least some of the speechinstructions as pre-existing, spoken navigation instructions.

Example 13 is a computer implemented method, comprising:

receiving a natural language instruction;

matching the natural language instruction with a pre-defined navigationmaneuver;

identifying context information corresponding to the pre-definednavigation maneuver, the context information being indicative of alocation at which the pre-defined navigation maneuver is performed;

generating a trigger comprising the pre-defined navigation maneuver andthe corresponding context information; and

surfacing the trigger.

Example 14 is the computer implemented method of any or all previousexamples and further comprising:

identifying any geographic limiter in the natural language instruction;and

generating the trigger to include the geographic limiter, along with thepre-defined navigation maneuver and the corresponding contextinformation.

Example 15 is the computer implemented method of any or all previousexamples and further comprising:

performing disambiguation of the context information and the geographiclimiter to identify geographic locations corresponding to the contextinformation and the geographic limiter.

Example 16 is the computer implemented method of any or all previousexamples wherein matching comprises matching each of a plurality ofnatural language instructions with a pre-defined navigation maneuver,wherein identifying context information comprises identifying contextinformation corresponding to each of the pre-defined navigationmaneuvers, and wherein generating the trigger generating, as a set ofordered triggers, a set of pairs of the pre-defined navigation maneuversand corresponding context information, and further comprising:

generating route information identifying a route between successivepairs of triggers in the set of ordered triggers.

Example 17 is the computer implemented method of any or all previousexamples and further comprising:

identifying a start point from the plurality of natural languageinstructions and from user-specific location information.

Example 18 is the computer implemented method of any or all previousexamples wherein generating route information comprises:

generating route information identifying a route from the start point toa first of the set of ordered triggers.

Example 19 is the computer implemented method of any or all previousexamples and further comprising:

generating navigation speech corresponding to the routing information;and

surfacing the routing information on a navigation display as a set ofturn-by-turn navigation instructions and the navigation speech on anaudio output device as a set of spoken navigation instructions.

Example 20 is a computing system, comprising:

maneuver identification logic that receives a plurality of naturallanguage instructions and matches each with a pre-defined navigationmaneuver;

context identifier logic that identifies context informationcorresponding to each of the pre-defined navigation maneuvers, thecontext information being indicative of a location at which thecorresponding pre-defined navigation maneuver is performed;

trigger output logic that outputs, as a set of ordered triggers, a setof pre-defined navigation maneuvers and corresponding contextinformation provided by the maneuver identification logic and thecontext identifier logic; and

an output system that surfaces the set of ordered triggers.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. A method performed by a computing system, themethod comprising: receiving a natural language input; generating one ormore natural language instructions based on identifying linguistic unitsin the natural language input and performing natural language processingon the linguistic units; identifying, based on the one or more naturallanguage instructions, an end point for a route and a trigger thatcomprises a navigation maneuver and corresponding context information,the corresponding context information being indicative of a locationcorresponding to performance of the navigation maneuver; generatingroute information that represents a portion of the route that includesthe trigger and the end point; and rendering the route information. 2.The method of claim 1, wherein performing natural language processing onthe linguistic units comprises identifying semantic meaning of thelinguistic units.
 3. The method of claim 1, and further comprisingidentifying, based on the natural language input, a plurality of naturallanguage instructions.
 4. The method of claim 3, wherein the pluralityof natural language instructions are indicative of a sequential set ofnavigation maneuvers to be performed along the route, and the locationcomprises a waypoint along the route.
 5. The method of claim 1, whereinthe trigger comprises a first trigger, and further comprising:identifying, based on the one or more natural language instructions, anordered set of triggers comprising the first trigger and a secondtrigger; and generating route information representing a portion of theroute between the first and second triggers.
 6. The method of claim 1,and further comprising: identifying a geographic limiter in the one ormore natural language instructions; and generating the trigger toinclude the geographic limiter, along with the navigation maneuver andthe corresponding context information.
 7. The method of claim 1, andfurther comprising: performing disambiguation of the context informationand the geographic limiter to identify a geographic locationcorresponding to the context information and the geographic limiter. 8.The method of claim 1, and further comprising: identifying a pluralityof natural language instructions, each indicative of a differentnavigation maneuver to be performed along the route; matching each ofthe plurality of natural language instructions with a pre-definednavigation maneuver; identifying context information corresponding toeach of the pre-defined navigation maneuvers; generating, as a set ofordered triggers, a set of pairs of the pre-defined navigation maneuversand corresponding context information; and generating route informationthat identifies a portion of the route between successive pairs oftriggers in the set of ordered triggers.
 9. The method of claim 1, andfurther comprising: rendering the route information as a set ofturn-by-turn navigation instructions.
 10. The method of claim 9, whereinthe natural language input comprises a textual user input.
 11. Themethod of claim 9, wherein the natural language input comprises a speechuser input.
 12. The method of claim 11, wherein rendering comprises:rendering a portion of the speech input in association with theturn-by-turn navigation instructions.
 13. The method of claim 12,wherein rendering a portion of the speech input comprises: obtaining asnippet from the speech input; and audibly rendering the snippet to theuser in association with a corresponding navigation instruction in theset of turn-by-turn navigation instructions.
 14. A computing system,comprising: a processor; and memory storing instructions executable theprocessor, wherein the instructions, when executed, cause the computingsystem to: receive a natural language input; generate one or morenatural language instructions based on identifying linguistic units inthe natural language input and performing natural language processing onthe linguistic units; identify, based on the one or more naturallanguage instructions, an end point for a route and a trigger thatcomprises a navigation maneuver and corresponding context information,the corresponding context information being indicative of a locationcorresponding to performance of the navigation maneuver; generate routeinformation that represents a portion of the route that includes thetrigger and the end point; and render the route information.
 15. Thecomputing system of claim 14, wherein the one or more natural languageinstructions are indicative of a sequential set of navigation maneuversto be performed along the route, and the location comprises a waypointalong the route.
 16. The computing system of claim 14, wherein thetrigger comprises a first trigger, and the instructions, when executed,cause the computing system to: identify, based on the one or morenatural language instructions, an ordered set of triggers comprising thefirst trigger and a second trigger; and generate route informationrepresenting a portion of the route between the first and secondtriggers.
 17. The computing system of claim 14, wherein theinstructions, when executed, cause the computing system to: identify ageographic limiter in the one or more natural language instructions; andgenerate the trigger to include the geographic limiter, along with thenavigation maneuver and the corresponding context information.
 18. Thecomputing system of claim 14, wherein the instructions, when executed,cause the computing system to: identify a plurality of natural languageinstructions, each indicative of a different navigation maneuver to beperformed along the route; match each of the plurality of naturallanguage instructions with a pre-defined navigation maneuver; identifycontext information corresponding to each of the pre-defined navigationmaneuvers; generate, as a set of ordered triggers, a set of pairs of thepre-defined navigation maneuvers and corresponding context information;and generate route information that identifies a portion of the routebetween successive pairs of triggers in the set of ordered triggers. 19.The computing system of claim 14, wherein the instructions, whenexecuted, cause the computing system to render the route information asa set of turn-by-turn navigation instructions.
 20. The computing systemof claim 19, wherein the natural language input comprises at least oneof a textual user input or a speech user input, and the instructions,when executed, cause the computing system to: render a portion of thenatural language input in association with the turn-by-turn navigationinstructions.